Non-Deterministic Segmentation for Chinese Lattice Parsing

Parsing Chinese critically depends on correct word segmentation for the parser since incorrect segmentation inevitably causes incorrect parses. We investigate a pipeline approach to segmentation and parsing using word lattices as parser input. We compare CRF-based and lexicon-based approaches to word segmentation. Our results show that the lattice parser is capable of selecting the correction segmentation from thousands of options, thus drastically reducing the number of unparsed sentence. Lexicon-based parsing models have a better coverage than the CRFbased approach, but the many options are more difficult to handle. We reach our best result by using a lexicon from the nbest CRF analyses, combined with highly probable words.